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Identif ication and Estimation in Path Analysis 
with Unmeasured Variables 

Abstract 

A variety of path models involving unmeasured variables are formulated 
in terms of j6'reskog*s (l 970 a) general model for the analysis of covariance 
structures . 
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Identification and Estimation in Path Analysis 
with Unmeasured Variables'^ 

A variety of authors (e.g., Blalock, 1969 ; Costner, 1969 ; Heise, 1969) 
have applied path analysis to problems involving multiple indicators of under- 
lying constructs. An important and often algebraically complex feature of 
such analysis is the determination of identifiability of model parameters. 

The purpose of this discussion is to demonstrate how a visual inspection of 
the path diagrajn can be used to simplify the identification question and how 
these problems may be formulated in jireskog^s (1970a) general model. 

I. A Single Factor Model 

Consider the case of a single underlying factor (F^) with three 
observed measures (X^,X^, and X^) as shown in Figure l.a. The factor 
loadings (p ) in this model equal the standardized path coefficients 

x.ii 

(b*,b*, and b*) , given the assumption that the residuals ^3 

are independent of each other and of the factor. It is convenient, though 
not necessary, to assume that both measured and unmeasured variables are 
standardized. For heuristic purposes observed correlations will be designated 
with ”r^* and expected values of these correlations by "p” . The expected 
correlations will differ from the corresponding observed correlations because 
of samplinri; and model specification errors. 



*The research reported herein f/ras performed pursuant to Grant No. OEG- 2 - 
700035(509) with the Uniued States Department of Health, Education, and 
Welfare, and the Office of Education. 
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Fig. l.a. A Single Factor Model 



A path analysis of this model yields the equations 

P, 



and 



12 



13 



^23 



b*b* ; 



b*b* , 

1 3 



b*b* 



( 1 ) 



Assuming nonzero correlations, equations (l) yield: 
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,, ^^2 ^12^23 2 

(bt) = — = Py p and 

2 Pi3 X^F^ 



( 2 ) 



O 

ERIC 



^ y f>i2 ■ 

Given only three observed measures the model is just identified, i.e., the 
observed and expected correlations are identical* With more than three measures 



p ,2 _ /•.*'|2 _ '^ i . l^ik 

Pv TP '^1 ' n 

^i^i ^ Pjk 



where i 4 j 4 k and 



(2a) 



4 



- 3 - 



ass\ming 4 0 • If there were a causal linkage (e.g., -> -> -> X.) 

from to X. then p would be the product of the intervening path 

± 1 X . 1' T 

1 1 

coefficients, i.e., the product of the path coefficients in the chain from 
to would be identified. If any loading exceeded unity, the model 

would be rejected. When there are m > 3 observed measures then the loadings 
will be overidentified. The number of over identifying restrictions is simply 
the number of distinct correlations m(m - l) 2 less the number (m) of 
p„ ^ to be estimated. Maximum likelihood or least squares estimates for over- 

XiFf 

identified models can be obtained using J&eskog's ( 1970a) general method for 
the analysis of covariance structures. We use path analysis only to study the 
identif lability problem, not for estimation purposes (Hauser 8 c Goldberger, 

I97O; Werts, Jtireskog, 8 c Linn, in press). 

The above analysis leads to our "rule of three": Whenever the correla- 

tions among at least three observed variables may be completely ascribed to 
the presence of an underlying factor, then the loadings (correlations) for 
each observed variable on that factor are identifiable. An important qualifi- 
cation is that the expected correlation between any two observed variables 
cannot be zero since equation (2a) would not be defined when that correlation 



was in the denominator. In practice, small expected correlations may lead to 
unstable parameter estimates, i.e., highly unreliable measures result in 
unreliable parameter estimates. 



II . Generalizations 

The Figure l.a. model with or without intervening, unmeasured variables 
going from F^ to is too limited for most causal analyses. Our purpose 

in this section is to consider other causal patterns which satisfy the "rule 
of three," i.e., in which the observed correlations among three variables are 
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nonzero and may be ascribed to the presence of an underlying factor. liq.uations 
(l), and therefore (2), would still hold if for one of the measures (e.g., ) 

and the residual 0^ of this regression of F^ on vrere indepen- 



dent of the other residuals and e^ , as shown in Figure 



l.b. 
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Figure l.b. 



If two observed measures influence F^ , 



X- 



F^ and X^ 



is no longer true that the correlation between these measures equals the product 
of the corresponding path coefficients, e.g., not in general equal 

b*b* 

□iD^ . 

Given that all residuals are independeni:^ when there is an intervening 

variable (l3_) between X^ and F^, the correlation between a pair of observed 

variables X^ and X^ will equal the product of the intervening path coefficients 

when X. I., <“ F, ^ X . , X. ^ ^ F., ^ X . , X, ^ I., ^ F, ^ X . , and 

1 1 1 1 1 1 1 j" 

X. I F.| <“ X . ; but not when two arrows point towards the same variable, 

1 1 j 

e.g., X. ^ F-, ^ X . or X. ^F. ^X. . In general the correlation 

^^illj illj 

between two observed variables may be stated as the product of the intervening 
path coefficients whenever the causal linkage between these variables does not 
include a variable which is caused by two other variables, i.e., when two 
causal arrows point towards a variable. To identify the loadings on a factor 
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we need to find three observed variables which are causally linked through 
that factor, the linkages satisfying the above criteria. 

III. Examples 

A. Our first example^ which corresponds to Figure 1 in Wiley and Wiley 
(1970), is shown in Figure 2. a. 




Figure 2. a. 



Tracing linkages for F^ • 



X 

X 

X 



1 

1 

2 



Fj^ — > F2 — > Fj — > Xj , 
^ Fj^ — > F^ ^ 9-ud 

F^ — > Fj — > Xj . 



Since these three linkages all include F2 and satisfy the requirements of the 

”rule of three” we may conclude that the factor loadings (p^ ) , i.e., the 

2 

correlations of each observed variable with F2 ; are identified. Thus, 









= , and 



2 " 2 






7 



(5) 




6 - 



The factor loadings on are not identified because the correlation between 

and cannot be completely ascribed to F^ . Likewise the loadings on 

Fj are not identified because the correlation between X^ and X^ cannot 
be ascribed to F^ . j6'reskog ( 1970b) shows that this model may be estimated 
by a single factor model with F^ as the common factor and that the example 
may be generalized to more than three measured variables. 

B. Out second example (see Figure 2.b*) corresponds to Figure 4 in Costner 
(I969)* The analysis is identical whether F^ F^ or F2 . 






X- 










X. 
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e^ 




Figure 2.b. 
Tracing linkages : 






X^ ^ -»x^ 



Xi ^X^ 



^2 ^ Fg Xj 






X ^Fg -^Xi^ 



(4a) 

(4b) 

(4c) 

(4d) 

(4e) 

(4f) 
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For F^ the factor loadings may be identified by linkages 4a^b,d or by 



e, i.e. 


, these loadings 




b* ; 


Xj^Fl 


1 




b^ ; 
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P 


btb* } and 




5 5 


P.. ^ = 


bfb* 


X 4 F 1 


4 5 



The factor loadings for F^ may be identified by 4b^c^f or 4d,e^f 
and: 

p = b*b* , 

p = b*b* , 

^X^F^ 2 5 ^ 

Since b* and b* are identified^ b'^' is also identified by these equations. 

The analysis may be complicated by assuming e^ correlated with e^ ^ 
in which case linkage 4b would not be valid, however the conditions of the 
"rule of three" would still be satisfied for F^ and F^ and all path 
coefficients and correlations between errors are (just) identified. Such a 
model would correspond to Figure p.a. in Costner (l969)« 

C. The next example, corresponding to Figure 1 in Blalock (1965)^ is 
shown in Figure 2.c. This model is basically a variation on the model of 
Figure l.b. 
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Figure 2.c. 

This model differs from that in Figure 2. a. in that F^ instead of 

F^ . The linkages are: 

— > F^ — > — > Xp > 

F^ Fp -> Fj Xj , and 

Xp <— Fp — > Fj — > Xj 

Since Fp is in all three linkages which satisfy the "rule of three," the 
factor loadings for Fp are identified and 



, (5a) 

, and ( 5ti ) 

(5c) 

Fj and cannot be ascribed to F^ 

the loadings on these factors are not identified. Our heuristic device would 
have been helpful to Blalock (I 965 ) since he obtained the equations correspond- 
ing to the linkages shown above, but did not solve them for the equivalent of 
equations 5a, b, and c. 



^X^Fp " ^1^^ " J^12^13 ■ ^23 

^XpF2 " ^2 = J^12^23 " ^13 

^^3^2 " " / l 3'"23 ■ "^12 

Since r^p cannot be ascribed to 
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D. Our fourth example, shown in Figure 2.d., corresponds to Figure 2 in 
Blalock ( 1963 )- 



X 



1 



e 



b- 






1 



b* 

°2 



X, 



I 




0 = residual of 



Figure 2.d. 

Tracing linkages which satisfy our rule: 



^1 ^1 ^2 ^3 ^ 



^ ^ ^ , and 



^4 ^ ^2 ^ ^3 



In this model it is assumed that is independent of X^ and X^ 

loadings on F^ are identified by linkages 6a, b and c and therefore; 

P = b* ^ 

XiF^ 1 

" x 2F3 _ = "^2 ^ 

^^ 3^1 " ■ 



(6a) 
(6b) 
(6c ) 
(6d) 
The 

(7a) 

(7b) 

(7c) 



It is not possible to find three observed variables whose linkages satisfy 
our rule for Fg , i.e., the linkage between X^ and Xj^ has two arrows 
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pointing at and the linkage between and does not include • 

Since it follows from equation (7c) that Px p * 

E. The fifth example^ shown in Figure 2 .e.^ has the special feature 
of two observed nonindependent variables influencing an unobserved variable. 
It corresponds to Figure h in Blalock (1969). 




Figure 2.e. 



0 = residual of F^ on X^ and X^ 
regression. 



When X^ is deleted X^ ^ X^ ^ and X^ form the model in Figure l.b. from 
which we conclude that the correlations of X^ ^ X^ , and X^ with F^ are 
identified. Similarly when X^ is deleted the correlations of X^ ^ j, and 
X^ with F^ are identified. Given the correlations among X^ ^ X^ ; and F^ 
the path coefficients b* and b* may be identified since: 



F " ^12^ F 

b* = 2 ^ a.nd 



1 - P. 



12 



b| = 



^X^F^ " Pl2Px^F^ 



1 - p 
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F. Our last example ^ shown in Figure 2.f. corresponds to Figure in 




Figure 2.f. 



From the analysis of the Figure 2.d. model may deduce that when is 

excluded that b* ^ b* ^ b* , b* ^ b^ ^ and b* are identified. Using the 

variables ^ X^ y and X^ we know from our analysis of the Figure 1 

model that the correlation of X> with F^ ) is identified and 

4 1 ^4^1 

similarly using ^ and Xg we know that the correlation of X^ 

with 5L (Pv TTi ) identified. Since the correlations among F-, ^ Fn ^ 

^ A4F2 1 2 

and X^ are identified it follows that the path coefficients b^ and bg ^ 
which are functions of these correlations^ are identified. As compared to 
Costner *s (1969) rather complex algebraic analysis of this problem, it may 
be seen that we are satisfied in merely knowing that the model parameters 
are identified. 



IV. Estimation 

j6reskog*s (1970a) general model for the analysis of covariance 
structures can be used to estimate the parameters for the models discussed 
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above. Werts, Jtireskog and Linn (in press) discuss the use of jtireskog*s 
model from the perspective of path analysis. Use of the associated com- 
puter program (Jtireskog^ Gruvaeus ^ & van Thillo^ 1970) for the present 
purposes requires the investigator to specify a matrix A corresponding 
to the factor loadings in factor analysis; a matrix 0 which is the variance- 
covariance matrix of the unmeasured factors, and a matrix 0 of residual 
variances. The matrices B and in J6'reskog*s formula are taken as the 
identity and zero matrix respectively. 

Consider for example the model in Figure 1 in which ^ 



A 



bj 

b| 

b^ 



[i] 



and 



0 = 



V 0 0 

0 V 0 

®2 

0 ' 0 V 



'3 



Define; X = column vector of standardized observed variables^ 
F = column vector of factors , and 
e = column vector of residuals. 

In matrix terminology; 

X = AF + e 



(8) 



Equation (8) is shorthand for the path equations (all variables standardized) 
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Xi = , 



^2 ^ ^ 2^1 ■*' ®2 ' 



X, = b*F^ + ej 



It can be seen that A is the matrix of the coefficients of • The 

parameters in the matrices specifying the model structure in Jiireskog's 
modelare of three kinds: (l) fixed parameters that have been assigned 

given values 5 (^) constrained parameters that are unknown but epual to 
one or more other parameters; and (3) free parameters that are unknown and 
not constrained to be eq_ual to any other parameter. In the abovo example 
the unity in <[> is a fixed parameter, whereas the b* in A and the V 
in 0 are free parameters . 

The expected variance-covariance matrix Z for this problem is : 



e . 

1 



S = A<[)A*+ b " 



(9) 



where the 1 in 0 is the variance of , for convenience standardized 

2 

(i.e.^ equal to unity) and 0 is a diagonal matrix whose elements are the 

error variances (v ) . Equation (9) should be recognized as a. shorthand way 

e . 

1 

of expressing all the path equations relating expected model correlations to 
model parameters, i*e.. 





1 


OJ 
1 — 1 
a 


^13 


E = 


OJ 
1 — 1 
Cl 


1 


P 23 




*^13 


P 23 


1 


Equation i 


^9) states: 





(where unities indicate observed 
variables were standardized ) • 
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1 = (b*) + V , 

1 



1 = (b|) + . 



1 = (b*) + V . 



; and 

This short description for a single model contrasts with the path analysis 
approach to estimation used by Costner (1969) and Blalock (1969) in the 
following respects ; 

(a) The matrix Z of expected correlations between observed variables 
will differ from the actually observed matrix of correlations because of 
sampling and/or model specification errors. Thus we do not use observed cor- 
relations in our equations as in the usual path analysis approach. Instead^ 
JiJreskog's program attempts to minimize the difference between observed and 
expected variance-covariance matrices using either a least squares or maximum 
likelihood approach. In large samples^ assuming that observed variables 

are distributed normally^ a chi square statistic is produced which measures 
the overall fit of the model to the data. Another way of gauging fit is to 
compare the differences between the observed and expected correlations gen- 
erated by the model. 

2 

(b) The degrees of freedom (df) for the X measure are equal to the 
number of overidentifying restrictions. In path analysis this corresponds 



'^12 



bfbf 



^13 

^2J> 



b*b* 



b*b* 
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to the number of different ways the path equations may be solved for each 
parameter. To compute the df it is necessary to count the number of distinct 
elements in Z (i.e.^ m(m + l) t 2) and subtract the number of parameters 
to be estimated {e.g., b*,b*,bt,V ,V , and V ) • There is no need to 
solve the path equations in Jtireskog’s approach, although the identif lability 
must be knovm. 

To analyze the model in Figure l.b., we merely need to note that when 

and are standardized the regression of on F^^ equals that of 

on and the residuals are identical. Thus we may use the same 

estimatim procedure for this model as for that in Figure l.a. (where = e^ ). 

Likewise the models in Figures 2. a. and 2.c. may be estimated by ignoring F^ 

and F_ and treating X ,X , and X^ as indicators of the common factor F . 

5 12b ^ 

The model in Figure 2.b. with the added feature of e^ and e^ corre- 
lated requires special treatment. The equations are: 



^1 = 
= 



= 



X,. = 



Fo = 



b*F-, + e, , 
> 

'3 ^ 



1 1 

h*p + e 
2 1 2 ' 



b*F2 + e. 



b*Fi + 02 



We know that b* is equal to the correlation between F^^ and F 2 so 
there is no need to replace F 2 by F^^ and ©2 in the first four equations. 
To specify a correlation between e^ and e^ , all residuals must be treated 
as factors, i.e., F'= (F^,F 2 , e^, 02 , e^, e^^ ) . The structure is: 
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A = 






0 

0 



b* 






b* 



0 



0 



0 



b* 



0 

0 



0 



0 



b* 

e~ 



0 



0 



b* 

6i 



and 



^ = 



b* 



0 

0 

0 

0 



bl 



0 

0 



0 



0 



0 0 



0 0 






1 

0 






0 

1 



0 0 



In contrast to previous formulations the error variances are standardized 

so that the correlations between e^ and e^ and F. and F^ are 

13 12 

estimated directly and in A the path coefficients of the observed varia- 
bles on their errors (b* ) are estimated. This model has 10 distinct 

e . 

1 

elements in E and 10 parameters to be estimated (b*,b*,b*,bj|^,b* ,b* , 

b* ,b* ^b*,p ) , i.e., the model is just identified. The expected 

®4 ^ 

variance-covariance matrix E = A$A' ^ i.e., the matrix 0 is taken to be 
zero. 

The Figure 2.d. model poses two problems: the parameters b* , bj|^ , and b* 

are not identified and the expected correlation between and or 

Xg is specified as zero even though the observed correlation may differ 
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from zero presumably because of sampling fluctuations. The analysis in 
Section II showed that does not contribute to the identification 

of parameters, i.e., only the product b^b* is identified with or without 
X^ . Without X^ the model is that of Figure l.b. and no purpose is 
served by retaining • Assiiming all variables are standardized 

Xi = substituted for F^ = t*Xj^ + as noted earlier. 

With F^ eliminated and knowing that only the correlation of X^ with 
F^ is identified the model may be written as : 

Xi = b*F^ + , (10a) 

, and ( 10b ) 

X, = b*b*F + b*bfX, + el where el = b*0 + e^ . (lOc) 

53513443 33 3 

For convenience define b*^ = b*b* and b*i = b*bf . For computational 

35 3 5 3^ 3 4 

simplicity define a new factor which is identical to the observed X^ , 

i.e., ^4 ~ factors are then F’ = (F^,x^) , 



A = 



b- 

b- 

b* 

35 

0 



0 

0 




) 



1 



$ = 



0 



0 




and 
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0 


0 
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11 

CVJ 
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CVJ 

0) 
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0 


0 




0 


0 


5 


0 




0 


0 


0 


0 






p 





In the fourth row of 0 the diagonal cell is zero to indicate the identity 
Xj^ = without residuals. If the expected matrix 2 is computed, i.e., 

2 = hC>A’ + 0^ we find: 





b*b| 




0 


b*b| 


^2 


b|b|5 


0 


1 55 


b*b* 

2°35 


^3 


bt) 

5i. 


0 


0 


"Si. 





This shows that the expected correlations of 
zero. This follows from the specification in 0 
with . 



and X with X are 
2 it- 

that x^ is uncorrelated 



In the analysis of the model in Figure 2.e., the correlations among 
Xi , , and F^ were identified first and then b* and identified 

from these correlations. The simplest estimation procedure is to estimate 
the correlations among , and F^ and then compute b* and b* 

from the estimated correlations . This problem can be handled by defining 
two factors x^ = X^ and ^2 ~ ^2 ’ structural equations are: 
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Xi 



^2 ="2 



X, = b*F-, + e , and 

b 5 1 5 



X4 = bJjF, + 



A = 



$ = 



are 


F'= (x^ 


’ ’=2 ’ ■ 


1 


0 

0 

! 




0 


0 
1 — 1 




0 


0 b* 
5 




0 


1 

0 




1 — 1 




Px F 

Vl 


PXiX 


2 2 


Px F 
^2 1 


Px^F 


Px F 
1 2^1 


1 



and 



0 ^ = 



0 

0 

0 

0 



0 

0 

0 

0 



0 

0 

V 



0 

0 

0 

V 



There are 10 distinct elements in E and nine parameters to be estimated 
(b*,bf,V^ ^ .P 






,V , and V ); so that the model has one 
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over identifying restriction. Note that the estimated elements of 0 should 

be used to estimate b* and b* v equal p _ ) • 

1 ^ “^12 ^±2 
In relation to the model in Figure 2 .f. Costner (1969) discussed the 

problem of ascertaining whether b^ was zero and of distinguishing the 

b^ = 0 model from one in which errors (e.g., e^ and ) were correlated. 

To see how this is accomplished in Jtireskog's approach^f irst consider the 

model when bg = 0 and treating residuals as factors : 

X'= (X^,X2,X^,Xj^,X^,Xg) , 






A = 



b| 

b- 

0 

0 

0 



0 

0 

0 



bl 



1 

0 

0 

0 

0 

0 



0 

1 

0 

0 

0 

0 



0 

0 

1 

0 

0 

0 



0 

0 

0 

1 

0 

0 



0 

0 

0 

0 

1 

0 



0 

0 

0 

0 

0 

1 
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$ = 



b* 

0 

0 

0 

0 

0 

0 



b* 

1 

0 

0 

0 

0 

0 

0 



0 

0 

V 

e 

0 

0 

0 

0 

0 



0 

0 

0 

V 

e 

0 

0 

0 

0 



0 

0 

0 



V 

e 

0 

0 

0 



0 

0 

0 



0 

V 

e 

0 

0 



0 

0 

0 



0 

0 

V 



0 

0 

0 



0 0 0 0 



0 

0 

0 

V 
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and 



y\$A' 



(i.e. , = 0 ) . 



Note that we have chosen not to introduce the residual 0 into the anaJ.ys i s 

because we wish to standardize both F. and Fp ; in VT-hich case jp = • 

This model is a variation of that in Figure 2.b. and all parameters are 

identified. There are 21 distinct elements in ^ and l4 parameters to be 

estimated so that there are seven overidentifying restrictions. To test 

bg 0 , we specify = FgF^ + Fj*F^ + e^ , i.e., in A the fouz'th ro\r ^ 

first column element is leFt "free” instead of fixed = zero. This model 

has one more parameter to be estimated and therefore six overidentifying 

restrictions. Thus the original model is more restrictive and will there- 

2 

fore typically have a larger X • In large samples, the difference in 
2 

X between these two models, with degrees of freedom equal to the difference 
in number of restrictions, can be used to test the hypothesis that 
b^ 0 . Similarly the model with e^ and e^ correlated 

("free”) in <[> instead of independent (fixed = O), would have six degrees 



of Freedom and the difference in X with one degree of freedom would be 

a test of the hypothesis that e^ and e^ are uncorrelated. A comparison 
2 

of the X for ht ^ 0 to that for p ^ 0 gives an indication of 

c 

which is the better fitting model. Costner (1969^ Figure lO) also raises 

the question of whether e^ and e^ are correlated. This hypothesis is 

tested by allowing the covariance between e^ and e^ in 0 to be "free," 
2 

the change in X with one degree of freedom providing the appropriate 

statistical test. Hypotheses involving "constrained" parameters may be 

tested similarly, e.g., b* = b^ (Heise, 1969) or V = V (Wiley & 

lb 

Wiley, 1970 ). 
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It caxL be observed that use of Jilreskog^s program requires the invest! 
gator to know the identification status of each parameter, but does not 
require the complex algebraic manipulations provided by Costner (1969) 
and Blalock (1969)* It is important to recognize the essentials of each 
model in order to fit it into J&eskog's general model. J 6 ’reskog's model 
assumes that the observed variables are "random” rather than "fixed” but 
it is doubtful that most applied sociologists need to be concerned about 
this issue which is minor in comparison to the usual questionable validity 
of measures and models . 
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